From 71b28568d98ff0cd56f70ef5eead118f70fef654 Mon Sep 17 00:00:00 2001 From: Julian Risch Date: Thu, 17 Oct 2024 14:51:53 +0200 Subject: [PATCH] Remove Gradient integration (#276) * Delete integrations/gradient.md * Delete logos/gradient.png --- integrations/gradient.md | 127 --------------------------------------- logos/gradient.png | Bin 10592 -> 0 bytes 2 files changed, 127 deletions(-) delete mode 100644 integrations/gradient.md delete mode 100644 logos/gradient.png diff --git a/integrations/gradient.md b/integrations/gradient.md deleted file mode 100644 index 67c4585f..00000000 --- a/integrations/gradient.md +++ /dev/null @@ -1,127 +0,0 @@ ---- -layout: integration -name: Gradient -description: Gradient is an LLM development platform that offers simple web APIs for fine-tuning, embeddings, and inference on state-of-the-art open-source models. -authors: - - name: Mateusz Haligowski - - name: deepset - socials: - github: deepset-ai - twitter: deepset_ai - linkedin: https://www.linkedin.com/company/deepset-ai/ -pypi: https://pypi.org/project/gradient-haystack/ -repo: https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/gradient -type: Model Provider -report_issue: https://github.com/deepset-ai/haystack-core-integrations/issues -logo: /logos/gradient.png -version: Haystack 2.0 ---- - -**Table of Contents** -- [Installation](#installation) -- [Usage](#usage) -- [Examples](#examples) -- [License](#license) - -## Installation -Use `pip` to install the integration: - -```console -pip install gradient-haystack -``` -## Usage -Once installed, you will have access to a Generator and two Embedder objects. -- [`GradientDocumentEmbedder`](https://docs.haystack.deepset.ai/docs/gradientdocumentembedder): Use this component to create embeddings of documents. This is commonly used in indexing pipelines to store documents and their embeddings in document stores. -- [`GradientTextEmbedder`](https://docs.haystack.deepset.ai/docs/gradienttextembedder): Use this component to create embeddings of text, such as queries. This is commonly used as the first component of query pipelines to create embeddings of the query. -- [`GradientGenerator`](https://docs.haystack.deepset.ai/docs/gradientgenerator): Use this component to query generative models with Gradient. This is commonly used in query pipelines to generate responses to queries. - -### Use the GradientDocumentEmbedder -You can use embedding models with `GradientDocumentEmbedder`` to create embeddings of your documents. This is commonly used in indexing pipelines to write documents and their embeddings into a document store. - -```python -import os - -from haystack import Pipeline -from haystack.document_stores.in_memory import InMemoryDocumentStore -from haystack.components.writers import DocumentWriter - -from haystack_integrations.components.embedders.gradient import GradientDocumentEmbedder - -os.environ["GRADIENT_ACCESS_TOKEN"] = "YOUR-GRADIENT-ACCESS-TOKEN" -os.environ["GRADIENT_WORKSPACE_ID"] = "YOUR-GRADIENT-WORKSPACE-ID" - -documents = [ - Document(content="My name is Jean and I live in Paris."), - Document(content="My name is Mark and I live in Berlin."), - Document(content="My name is Giorgio and I live in Rome."), -] - -indexing_pipeline = Pipeline() -indexing_pipeline.add_component(instance=GradientDocumentEmbedder(), name="document_embedder") -indexing_pipeline.add_component(instance=DocumentWriter(document_store=InMemoryDocumentStore()), name="document_writer") -indexing_pipeline.connect("document_embedder", "document_writer") -indexing_pipeline.run({"document_embedder": {"documents": documents}}) -``` - -### Use the GradientTextEmbedder and GradientGenerator -You can use embedding models with `GradientTextEmbedder` and generative models with `GradientGenerator`. These two are commonly used together in a query pipeline such as a retrievel-augmented generative (RAG) pipeline such as the one below. - -```python -import os - -from haystack.components.builders.answer_builder import AnswerBuilder -from haystack.components.builders.prompt_builder import PromptBuilder -from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever - -from haystack_integrations.components.embedders.gradient import GradientTextEmbedder -from haystack_integrations.components.generators.gradient import GradientGenerator - -from getpass import getpass - -prompt_template = """ -Given these documents, answer the question.\nDocuments: -{% for doc in documents %} - {{ doc.content }} -{% endfor %} -\nQuestion: {{question}} -\nAnswer: -""" - -os.environ["GRADIENT_ACCESS_TOKEN"] = "YOUR-GRADIENT-ACCESS-TOKEN" -os.environ["GRADIENT_WORKSPACE_ID"] = "YOUR-GRADIENT-WORKSPACE-ID" - -retriever = InMemoryEmbeddingRetriever(document_store) -prompt_builder = PromptBuilder(template=prompt_template) -embedder = GradientTextEmbedder() -generator = GradientGenerator(base_model_slug="llama2-7b-chat") - -rag_pipeline = Pipeline() -rag_pipeline.add_component(instance=embedder, name="text_embedder") -rag_pipeline.add_component( - instance=retriever, name="retriever" -) -rag_pipeline.add_component(instance=prompt_builder, name="prompt_builder") -rag_pipeline.add_component(instance=generator, name="llm") -rag_pipeline.add_component(instance=AnswerBuilder(), name="answer_builder") - -rag_pipeline.connect("text_embedder", "retriever") -rag_pipeline.connect("retriever", "prompt_builder.documents") -rag_pipeline.connect("prompt_builder", "llm") -rag_pipeline.connect("llm.replies", "answer_builder.replies") -rag_pipeline.connect("retriever", "answer_builder.documents") - -rag_pipeline.run( - { - "text_embedder": {"text": question}, - "prompt_builder": {"question": question}, - "answer_builder": {"query": question}, - } - ) -``` - -## Examples -You can find a full code example showing how to use the integration in [this Colab](https://colab.research.google.com/github/deepset-ai/haystack-cookbook/blob/main/notebooks/gradient-embeders-and-generators-for-notion-rag.ipynb). - -## License - -`gradient-haystack` is distributed under the terms of the [Apache-2.0](https://spdx.org/licenses/Apache-2.0.html) license. diff --git a/logos/gradient.png b/logos/gradient.png deleted file mode 100644 index 546148134175b4395adadd92a3cebc914a1f0bed..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 10592 zcmeHt2UL^mwqS@vibtiYfFROEnn*L!M7jp08ajfJ-U$Sd5=2ExI7pSQNNAx%s!~-T zD$)c3ger)X5TqD-c|YJi_spF)Yt343y_s2eO|p{z%l>xX``cg3)7yGl#~C>pK_Jj^ z9c^_35Qv5de7fn60+9Z5DHFhh*8Upo8VH1sJ+^%p0{rH+(>8#CKmkG^P-r*^v2LKFfsSwlT<1Ov1`-9@^dJZg3+M>Yq5<9@8cxtYH=qU5 zrQ!OkZ9pUP8wU&mJ#hrl{>Cu_p8HL){ja~CN3v=D$uS%J4{RD@_L0BZ-TQRR4Bg*= z2ZM*UsW%8D!N32ZLFKCWfQ4t#CL!J1={;yNj}~K6dz|LJ~-?)?{#-C1h1Tgl!O$YG9xc9ucFtz`%nXQ z&EMp}FC{()A0H2>q@9K33o4h$xBzRT#=NLmXwwj2Qb9F{oQ=-qQu?2`45@= zgO57G+s@0;!^hFxjd!2#U0Zjgj}jl>zM#KehkE)r-v6g0H}BtO0R~F$_efrnkdpk* z$Pg&U{{`88&mq}?U5DZn_nARW4G`Y$F35eglrKw3DISRUAE5tK@Q~u41jcTTKFZR+ zN&W@=uT=1V_Ww=J-=Y6Xam&jQ0l4x&vrE5){2lf;{C)eNw;fRk7r45kE5glt-x?Wt zz?%QG=U*wWySuo18G78cLns4+4oLn6`d91&Ii`P;BPaWJIe+W{DD1R56+-ZL*y^?{hseKWB^GBq#~ z&EI*`6%~(P6!k2<9~KSAbt8ge&J-1vZCUb5LEZE0}LBlu&}VO{1YR)Qj9G0Ym~^V z`p(JVQ-1OXUX;3z-pU__4X0UNC1pT~Gq+DO|MDJ2V$tK8`;LxhbGSO#+5Jwgv~RZ$ zQ<*mBdSvFdwc#(G{TSTT8Fx1FW#*nAsnXr6qqMA6&KRi>J~U327#^oRWZ%W`AcCOs zGHbg6c8?q(YvN`*HCF99E_)ckas6>o&5|gBJ$~^zq0n&TJO1WDbVu1xU*9M{XIyw( zsafdGX@))sxgz5>&(5^%&b((cr(`A}Dg-ySsC}3X^J0tQM8#OgZ&N=as9w|U#lPV$ zjCE1Mu&wuF{6shUtp!Xl4zo@U;)6@$Ok&tizNR=Z#b{B~wxaRA7yn^525=Vf3BTPy`eBOB&=MON zkE$K2D8ST(7gA*lJLtjzjGP%_58}g%4#mjQykt5ImUx#hjuN8g_nRdPeFZn!GE713 zcfOwoy5$89vmY%3rXX>7%yXqO-_*|^s1Sc@ebsy0khm%-X$eccHg%w>DkqvuEe$K} zX=Z(NApNG<`0~#XpHY_Pghgqd4ie(GS8UwTTPcvtVl5nns97AyMLN`6k^3))~CSIRH04;#;i7{7J`S2VPsuC2Q?9V36^XKkv@}ICkhP5LzC> z)2J7e&gdrZ2_vU@m>dq@PDn$G1J!3|V$#TCORWwc`J{-N#&e+c0f|--CG{4R=++~A zv5upI@z6yiaB^B-h%|d$Fh4eYD2`l9^*;1uXp;$G042kU z-)fyakR93-3vQ2BKBUjZJF%{ybSbaEjomkehv{#o0!VNKS`?xPz1fhF@ z2Z53Lv#IowttGelYA4|++Lk>Dp#L=E{Zv_$ri0+Xx0VHPJgUJW4N21bwJ1U8&v{Z` zdn95OUJFib4-;cOue-EZ&l%lVkuYb>50?1-L}6hNsP{!)&U4{6>925;HW%9W z0)T0;agPkI4KEkWJf?23@3xe1)%c>ML2Ot*LoF65{w;2wnvq#Mm@;?2-YXV9^g6X2 zEPf~n4M|O(TDxSMp#ibzNiAn@Ip7*Z`^>ubtto1i+1U?9?r&H72R7|%3jm%x&+U9? zVCw#h-_=nSJh*qne6?WwR7owCT8d_3zj5F|Gqg*8iQDq3v@NB?v7RJ0a9}gjj4AJy zLuK(TNp5Gl{u_@1+yq+or+QT-aQy@E!8W5s(y(mt?~2(~-W2pDvtv`Fyp!44{9F!E z+02+@-=~Wf*~#5b<_g&H$pL=ZDTh)^ z%aW|`!Slc<7wsAHP?XPH-{*Auk!q>+^;IbQ@NzA-GBV@#GXQ!fN>vij-Il6s8}n^l zl%)wAZOm|c@3 zUTO!?H{++g&rO{*Yn6QBHn8G5tv^c0Y%?161dmN*Be~Co_IciQf$_$w&S&K@Z&>@k zj7kL-5O(tdFGBQK1VjELZ)=F0$2ZkR_{Aw;J!lOW#SD(J7RjS0wjPRN!-jrDmbV!z zR?s5rW>EL@?V3phteFrB65BbrIjQz1zP0d^<<9CPhnlNtL(_& z?9_tsDy0cAZ;q4pl-K66VB?!3MmZO{1;CzSrW(A~;)31t;vf4{jw=5wuHa`ZbZjP> zlt-2)@{{bM(=uDGxhS_r78quL@hVp8KcIb^simotA+FqHWYAVv^mUtSXw_@rRYDl2 zU93^zmEAji_*R)FY0htLJjd_$Nz+Q$CU}@>ioQ;Hty+C9IQ{jtTx4V98&TegP^`gL zw&YKnj3nqORY(Wj;Kgth!3lS?-LEB5gJpr?_qq>i>q_o&F=_T<2itpkq}_=t5YQ52 z+8WU?(#|upE|R0>ZubC-22S(hh4jxfac21hk88t56I zg&Zq*9vW{=<`VO-vRbyj{HbU5XMh{!d45M_=4v6dM?r5`1EOqX`E*urPaMOXS7}l^ z$LuT*^RT?Q8@{r8hOTxnoLoAm#Ne~_T(462XLySYgB_Kpa1K-1Q}~u9z3KuO~Upt%i^CA^T3DF;L7$byfQK;SZMDaUQz-;o> z?hL+)>L_;+@rDsyt}v6a@0%jFh@GMgcF*^g)?``0H!^nIWW$r4o9t4s{?fHxUFWMN z)tj`bn+CzDOCrE95p5Zqg(+?mC2EcNZ&M}r3iUH>UV17j#5nEAzANmvn$owpB$dSM zTzgrz@~1xQmp#j~pwq0lYSHP_QfBS$={ey-T3rox7$BoFUr0hl34T>~)F-Iu#eg5JbpI7voJjdhSgZHP;f3r&Zid{j6 zt0dFm)1VgWX$<%7Um1nvY;LM#hm=;j3BDQkD4@aTd{BYN$ zic@=BS@~5t`m@P=6Sx~+!uQDUNZyO}9a^`^%U8$T<2UwRrerT-;S5p~>{D zh2&#Vi#Hz_)N3VQ`ee*`z&sp2wZGsZ)@5KSe-P8|FF;Kw$}se%(~h`7Ly071@^YpN zfwPB|Qt>r;l0jp}bbRehefp{ecH$^o#A>F-(mA@D*SeoDSojeJKU9^Viaz<`EYoD! zt49(H<()E2$j(2ZL$w8$DU;=y^RdZ9A5Xbqj75-N`O$*OCFgX8tp2z{u~_zq&KYUj zbZ#!fYCYpH!QO}H{1n12M`d;%VuM|yoKGB~Rrcw5SX=B`>;!q4F~lm8;S?l3(3|Y_ zHX4b85`6*<8{?Fz=ILLLvUGJEd3Vm##rylJvAgmHP@kz2xOE90WuuX2JvV~C8o~@6 z7uH=$UY0Zz78Jd6x8p=v-TCZTE^E6JYagFvT){)zF_9hEef>)08EsyYAF~G8h;CWs z<-m^!S1o)2hN{wzQi>#|TM7E$Cf&R#=Js29yQo6D7;bs2c+%2%DsfY}xsj0nqoSb# zv7Mw^K^7x!tP=x9CH2NdF?mH3ujj6u=O+6te#$}w<>fogzASVRtSt)l9WgG%QO|1S zC~c!0?ee0_zZ)x3GddpYK%z<nIx>Dj}hZ1B1_kJ)wEJn9}0MT6^p{c&$crXzFcveXzAcWn+;7jt>q zpS$4m@*C9$KNcn`Pj*_b)j6QI*1xnzc6fw-kA_=o`kh*R4J-J$eTr_{wBXYxG6L;T zy@I-vw<)b(@KZt0&5?tfd@5lfw=H>WS??VLVd(+aMZj;Nai@Sy`|l@D$yw z|I??R9BWl8`Zelu3`aT)my0m5k`)*X6_T5n;-TTcHn0#nZpIOy6w(UFiCNPIxb#XYh+>7hE%oi%z(!QsoM zxYygek6-8Vto$agJ}?FQ^MuJl$G}eI^X3wzaPQVk(3RD_H2O*DG}cP{61;qR+b)Z{kQ zqSAGmWQ*zEYvw^?VaP(d&Eqwumu&`f#LWn)rH+)c4Hs`stM7j(PYevtxeQ~0fOe3K zJX%+umao^+@13_}R?9oRcAyRp~!r2giEqTPl%T%5Kc6)k&g!@LgfH z@q%3i^rA>scP?xOx1=0{Q#LHnyPOzVDExoFxYS8_EN>2twu=KB(Xwdi4zFbXU<344e0;p= z*9Zt*RFDt*Z2d(Aje;E!53a45Z9;PW*C$P#L2NIh??5E1gZk418PhfFFH2s+_g>LZ zlH|kc$;!IyqF}px35(A{F;v0lyH<@0r6PiYY;b{SW3T*z-tzBq3Ig(k#Ou8*t~tOE z@^?r{DKTxWvOX~~>idwvqS;}=r|fH>NKMfgfhW>H-3+MQCAhDT*<8=jaFdnIHi$^y z=bdtQzWWCJ;=8k`HX*_I={n~YgF@!~bm9Y76)Ro60C>|G?h5;qhBj!ldw#K--94#u zg^R;et;+tV5f46EV%Bva3wBxQf-xckqd-Lpj7JPLnckNpxu8R$I_#6gH$%}mR@3@* zddMa@g)#?1+>vhvZdf~5fxiW=ZBnk9stS%me{Q#&hm#YBTxZrX{MV~h6Wm2>LN(!e zOgZ5QG2l??xB+2OByZH#+5*y8Dl|7h*KZme<2b%TWsUv7a0Y+EzOne8ecF)G=U|E$ z^o9T6g@t6x-f6z7u7U-E$$Jv!oM468_iQ`ySLff#l(gw;$z5d>^R*~V>T5O0G8>L< z?^&9Jzf&I~!&$Fk=~;Ue+9JHu_>#`lGe!fG-y$H?+p51}eBNiSbwctnmU(NBMK{ zKlX1}+)>Bu{FKn$?7HQst0S-|zFO~#eOJsx!%tM>w9U|z&+x7fUchsr$6gwy-ifCq zsn>2J3(Z{^Lkcb^Hh#*@vlB1>6m(R zLUM=CRE{?ilreWeR7_(VW3ixgnLe3|8w03)%CvAlICV?dOW6bnCSi)2D@KM z{{W*mx1G2a6Ag5#ST>tnV;74&MS!Kjb0 z)=uJEdmU(>(!0;*Bb7x{m7m5qX5q(NQzVM9^S|z`MXGKEO{I9>GnT|@KbpOn?OF}j8~v4nV%A{V)l)UYfycQ1kgyR&Eea_pqEk*-wG!>0Xq5@l zfFZ|*2{Mbg@$FZuVWG^xN-(j`rPoa7x=wr(4Op{?E<3U8DYG*}v4_nG;1IV38^Tlb zrk$Qi#C$v`8lH^E6RC9tiolSOsDJTRS5UElr6UV(S#8tljJXAPZgGK~c(Dd>w(irt zd%ueJ_S?`>e_i(%ca|@+Z!_v0cW?1obo{om zYwKYUFs&0?O{{6YY0T5MjatAczbI02D(NC`2$(k7givJqNHyDfWqqrU7>! zuaq$wIHR`It6a7#sAeW>n@KB2`{}dyS72T#8QVPrS3~H0p=KEN4F_fzV72BGFn+MvExYGG3^}G3Zs9YW7j|)|uiCCv79l zf8RO8AnkVXxJ$(p?ZR%xu>Tc~PvK3kAPd{VX`a=4l9MHwBqTU*Y2BTZmY(hWqBFCp zHeXt32v>4UReF=BZ(X#@b{5eqRb8luy++!MJ7kg!Cycd{Iju@hE3)uPrG$2Vv-!2d zK7Wi(dSDk8CKW^C!t3Q^^jG>+)g>)zXNS;aHSV_4UFeQLGOgZa zi~%m#61E9axfc_gzC%U{5y&h4j~GBRaN(|xX~xkVNRG}lUX{1DDN#?h)>9uc1CMgTj6Qp_P%Hpp<{e*=!v_ITYUX@kY1v;< zBB;pnQPkrcaM4R-p_SBYim0p+niW59-(9bN(cwK<*`C$q0Y8yJF2EeoU!hm=&A8V6 zAk9`S^Oqm9ci~TvRUBqZ9Jv#$zfGR@pvDR?ErBSmK0aZ}nCnwpy}|0$f^- zo>~mSjLD@ob3%zaGAkX4F1}Vhk+k%RryVjI$2VjkH_dOBk5m((*oU_!V-Uk*u9okj zXzA;Kx$S-ssWX-fofQqI(8hv2ZQw1edn(KN&0ur}BsHVIVqylkug-CwsJ}QcU;pSh zJ(B|y6Z}{Tqty-61GdEwvvU{3mCF6OiIYOG5i*WmZdM z`|t~*vh`GnygLY4p|a}GQrM~x*WOWV7?87ceP#O>OAqC$n9+{(HNHqrN}Zp0417%} zt@Y@o`pU>uD_BmIlg>CVP=+23v^K_JklU^gZXV}Wz;u^W1|Sjyt83J&H-Y51>lQyK ziVdHRFTh2z3|!~Ek91pwj}-oIhmf~4)EY8P<%a!Q=zq7;ZKSP1e^$t2vcKucJD6jU zHP*-!siZqDQ?)y-&+KwDJD8^5w}0&MptDVLY6eOX(wvXvsJy~;E2cLbaLn2zvkXe* zqeS5gbDJ9N{Ngr4aPB*Pd)7>!a4VaFb z>h&My1U6QUiYwY+kT9dMRUAywYiT2{)Iaw9-}s`(>TqHyJ| zy0HN}xRhZS*y4n^LVhw2uyx_xoZzE6yilSkZosz$8}7$aZqNc=i`#nB?bIT!pgM;Q zqf;%d|ETa$Ea${ZP4(E-H_T^>RBJ=5XN(A>x{bM9yLS7lb00m+9JO#OHDn8|k+1Q^ z7FP{2{I)jkF&k}50qfekZdHuvh3v~wE#|zmngWZ0b6@rLs)S?RfT^zwzW}@c5QbJ* z$4w~5_U|(L<@CeOwpB7zWq2+VSB^1+j|qVNpanl!=*XbU6V)tJ&uc6 b=(klUFBeC2u~(Y+|2L?kp{HJ=W)t=wgLL>g