如何构建用于查询Redshift数据库的前端(希望使用Rails)

所以我有一个带有足够表的Redshift数据库,我觉得值得花时间构建一个前端,以便比仅仅输入SQL命令更容易查询它。

理想情况下,我可以通过将数据库连接到Rails应用程序来实现这一点(因为我对Rails有一些经验)。 我不知道如何将远程Redshift数据库连接到本地Rails应用程序,或者如何使activerecord与redshift一起工作。

有没有人有任何建议/资源来帮助我开始? 如果预先制作的选项比Rails更容易,我可以选择将Redshift数据库连接到前端。

#app/models/data_warehouse.rb class DataWarehouse < ActiveRecord::Base establish_connection "redshift_staging" #or, if you want to have a db per environment #establish_connection "redshift_#{Rails.env}" end 

请注意,我们在5439上连接,而不是默认的5432,所以我指定了端口另外,我指定了一个模式,beta,这是我们用于不稳定聚合的模式,你可以为每个环境提供不同的db,如上所述,或者使用各种模式并将它们包含在ActiveRecord的搜索路径中

 #config/database.yml redshift_staging: adapter: postgresql encoding: utf8 database: db03 port: 5439 pool: 5 schema_search_path: 'beta' username: admin password: supersecretpassword host: db03.myremotehost.us #your remote host here, might be an aws url from Redshift admin console 

###选项2,直接PG连接

  class DataWarehouse < ActiveRecord::Base attr_accessor :conn def initialize @conn = PG.connect( database: 'db03', port: 5439, pool: 5, schema_search_path: 'beta', username: 'admin', password: 'supersecretpassword', host: 'db03.myremotehost.us' ) end end [DEV] main:0> redshift = DataWarehouse E, [2014-07-17T11:09:17.758957 #44535] ERROR -- : PG::InsufficientPrivilege: ERROR: permission denied to set parameter "client_min_messages" to "notice" : SET client_min_messages TO 'notice' (pry) output error: # 

更新:

我最终选择了选项1,但现在使用此适配器有多种原因:

https://github.com/fiksu/activerecord-redshift-adapter

原因1:ActiveRecord postgresql适配器设置client_min_messages原因2:适配器还尝试设置时区,红移不允许( http://docs.aws.amazon.com/redshift/latest/dg/c_redshift-and-postgres- sql.html )原因3:即使您在ActiveRecord中更改了前两个错误的代码,也会遇到其他错误,这些错误抱怨Redshift正在使用Postgresql 8.0,此时我转向适配器,将重新访问并更新如果我后来发现了更好的东西。

我将我的表重命名为base_aggregate_redshift_tests(注意复数),因此ActiveRecord很容易连接,如果你不能在redshift中更改你的表名,请使用我在下面注释掉的set_table方法

 #Gemfile: gem 'activerecord4-redshift-adapter', github: 'aamine/activerecord4-redshift-adapter' 

选项1

 #config/database.yml redshift_staging: adapter: redshift encoding: utf8 database: db03 port: 5439 pool: 5 username: admin password: supersecretpassword host: db03.myremotehost.us timeout: 5000 #app/models/base_aggregates_redshift_test.rb #Model named to match my tables in Redshift, if you want you can set_table like I have commented out below class BaseAggregatesRedshiftTest < ActiveRecord::Base establish_connection "redshift_staging" self.table_name = "beta.base_aggregates_v2" end 

在使用self.table_name的控制台中 - 注意它查询正确的表,因此您可以根据需要为模型命名

 [DEV] main:0> redshift = BaseAggregatesRedshiftTest.first D, [2014-07-17T15:31:58.678103 #43776] DEBUG -- : BaseAggregatesRedshiftTest Load (45.6ms) SELECT "beta"."base_aggregates_v2".* FROM "beta"."base_aggregates_v2" LIMIT 1 

选项2

 #app/models/base_aggregates_redshift_test.rb class BaseAggregatesRedshiftTest < ActiveRecord::Base set_table "beta.base_aggregates_v2" ActiveRecord::Base.establish_connection( adapter: 'redshift', encoding: 'utf8', database: 'staging', port: '5439', pool: '5', username: 'admin', password: 'supersecretpassword', search_schema: 'beta', host: 'db03.myremotehost.us', timeout: '5000' ) end #in console, abbreviated example of first record, now it's using the new name for my redshift table, just assuming I've got the record at base_aggregates_redshift_tests because I didn't set the table_name [DEV] main:0> redshift = BaseAggregatesRedshiftTest.first D, [2014-07-17T15:09:39.388918 #11537] DEBUG -- : BaseAggregatesRedshiftTest Load (45.3ms) SELECT "base_aggregates_redshift_tests".* FROM "base_aggregates_redshift_tests" LIMIT 1 # { :truncated_month => Thu, 31 Jan 2013 19:00:00 EST -05:00, :dma => "Cityville", :group_id => 9712338, :dma_id => 9999 } 

祝你好运@johncorser!

本教程可帮助您使用redshift适配器设置rails应用程序:

https://www.credible.com/code/setting-up-a-data-warehouse-with-aws-redshift-and-ruby/

简而言之:

克隆示例应用 :

 git clone git@github.com:tuesy/redshift-ruby-tutorial.git cd redshift-ruby-tutorial 

通过~/.bashrc (或dotenv )设置ENV变量:

 export REDSHIFT_HOST=redshift-ruby-tutorial.ccmj2nxbsay7.us-east-1.redshift.amazonaws.com export REDSHIFT_PORT=5439 export REDSHIFT_USER=deploy export REDSHIFT_PASSWORD= export REDSHIFT_DATABASE=analytics export REDSHIFT_BUCKET=redshift-ruby-tutorial 

在Gemfile中使用gem activerecord4-redshift-adapter redshift activerecord4-redshift-adapter

 'activerecord4-redshift-adapter', '~> 0.2.0' # For Rails 4.2 'activerecord4-redshift-adapter', '~> 0.1.1' # For Rails 4.1 

然后您可以像使用普通AR模型一样查询redshift:

 bundle exec rails c RedshiftUser.count 

(披露:我还没试过这种方法,但我可能会很快)

您可能需要考虑http://www.looker.com/ 。 它是探索数据库的前端,允许轻松可靠的查询和业务人员也可以使用的GUI。